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ABSTRACT 

Project CELSIK (Career Education and Learning 
Strategies Implementation Model) had as its primary goal the 
provision of supplemental bilingual special education with a 
particular emphasis on the acquisition of learning strategies. The 
learning strategies approach was designed to improve cognitive 
functions, infuse career awareness, and help students understand the 
cultural dimensions of the workplace, as well as to identify specific 
areas of career interest. The program included computer literacy and 
extracurricular activities such as trips and guest speakers. The 
project served 137 Spanish-speaking special education students of 
limited English proficiency. Project CELSIM met its objectives for 
staff development, counseling, career education, and Native Languane 
Arts, but failed to meet its objectives for content area subjects, 
attendance, or curriculum development. The 

English-as-a-Second~Language objective could not be assessed. This 
evaluation report describes the evaluation methodology, the project's 
implementation and attainment of objectives, outcomes, and 
conclusions and recommendation. (JDD) 
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CAREER EDUCATION AND LEARNING STRATEGIES 
IMPLEMENTATION MODEL 
(PROJECT CELSIM) 
1988-89 

SUMMARY 

Troject CELSIM was fully implemented. The project 
provided special education students of limited English 
oroficiency with supplementary instruction in English 
as a Second Language, Native Language Arts, content 
area subjects, courses in computer literacy and career 
awareness, and counseling. It also engaged in staff 
and curriculum development activities. 

Project CELSIM met its staff development, counseling, 
Native Language Arts, and career education objectives. 
It failed to meet objectives in content area subjects, 
attendance, or curriculum development. It was not 
possible to determine whether the project met the 
English as a Second Language objective. 

Career Education and Learning Strategies Implementation 
Model (Project CELSIM) was in its first year of a three-year 
Elementary and Secondary Education Act (E.S.E.A.) Title VII 
funding cycle. Project CELSIM' s primary goal was to provide 
supplemental bilingual special education, with a particular 
emphasis on the acquisition of learning strategies. The 
learning-strategies approach was designed to improve cognitive 
functions, infuse career awareness, and help students understand 
the cultural dimensions of the workplace, as well as to identify 
specific areas of career interest. The program included computer 
literacy and extracurricular activities such as trips and 
appearances by guest speakers. The project served 137 Spanish- 
speaking special education students of limited English 
proficiency (LEP students} at James Monroe, Theodore Roosevelt, 
and William H. Taft High Schools in the Bronx. 

Project staff designed r rograms to promote growth of 
language abilities and career awareness. An important aspect of 
these programs was to help teachers incorporate learning skills 
instruction into the standard subject matter. A primary focus of 
Project CELSIM was to provide counseling to participating 
students . 

Project CELSIM met its objective for staff development, 
counseling, career education, and Native Language Arts, but 
failed to meet its objectives for content area subjects, 
attendance, or curriculum development. The English as a Second 
Language objective could not be assessed as stated, because data 
were inadequate for this specialized populatior . 



The conclusions, based on the findings of this evaluation, 
lead to the following recommendations: 



Make objectives more realistic for the targeted 
population. 

If funds permit, increase the time devoted to the 
program by the career and curriculum specialist at 
William H. Taft High School. 
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I. INTRODUCTION 



This report documents the Office of Research, Evaluation, 
and Assessment's (OREA's) evaluation of the 1988-89 bilingu?! 
special education program, Career Education and Learning 
Strategies Implementation Model (Project CELSIM) . Project 
CELSIM, funded by Title VII of the Elementary and Secondary 
Education Act (E.S.E.A.), served 137 Spanish-speaking special 
education students of limited English proficiency (LEP students) . 
The Bilingual/E.S.L. Unit of the Division of High Schools 
(D.H.S.) and the Division of Special Education (D.S.E.) of the 
New York City Board of Education operated the project at James 
Monroe, Theodore Roosevelt, and William H. Taft High Schools in 
the Bronx. 

The program provided students with supplemental services in 
English as a Second Language (E.S.L.), Native Language Arts 
(N.L.A.), and the contend areas, using a learning-strategies 
approach to enable students to maximize existing academic skills. 
The program also helped students prepare to enter the work force, 
offering computer literacy instruction in its career-oriented 
component. The program also proposed to develop learning 
strategies materials for students as well as to train teachers in 
this approach. 

HISTORY OF THE PROGRAM 

Pro}ect CELSIM completed its first year of a three-year 
cycle in 1988-89. Its main goal was to provide programming 
designed to meet the specific needs of bilingual (Spanish) 

© :* 
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special education students. These students had not previously 
received Title VII services and were historically a greatly 
underserved group. Project CELSIM was implemented at James 
Monroe, W^'fliam H. Taft, and Theodore Roosevelt High Schools in 
the Bronx. These three schools had large concentrations of 
Spanish-speaking LEP special education students. 

SETTING 

The area surrounding William H. Taft High School in the 
South Bronx is very dilapidated. Boarded-up and abandoned 
buildings are within sight of the school. James Monroe High 
School, also in the South Bronx, is in a slightly better 
neighborhood. Theodore Roosevelt High School is in a more 
attractive residential area of the North Bronx. 

PARTICIPATING STUDENTS 

Approximately 20 percent of the targeted students had been 
in the United states for less than two years. In addition to 
having identifiable cognitive/language needs, they were 
experiencing the difficulties of adjusting to a new culture. 

These students were very low achievers, typically scoring 
below the tenth percentile on achievement tests in reading, 
written language, and mathematics. Half were over-age for their 
grade level, and all came from low-income families. Students 
were hampered by illiteracy in their native language, 
responsibility for sibling care, and pregnancy. 

2 



STAFF 

Project CELSIM's Title VII staff included a director, deputy 
director, project coordinator, guidance counselor, staff 
development specialist, and four career and curriculum 
specialists. The principals and assistant principals of special 
education at each site aided the project staff. A bilingual 
guidance counselor visited all sites but did not have designated 
office space at any of the schools, making it difficult to meet 
with students. 

The career and curriculum specialist at James Monroe High 
School had been a paraprof essional for eight years before 
becoming a teacher in the special education department. She held 
two master's degrees, taught N.L.A., and had been instrumental in 
setting up the bilingual component of the special education 
department. 

Two half-time career and curriculum specialists served at 
Theodore Roosevelt High School. One had taught both special 
education and mainstream students and was expecting to complete 
his master's degree during the summer of 1989. He taught 
computer literacy and art education and was manager of the 
Comprehensive Competencies Program's (C.C.P.'s) Learning Center. 
The other career and curriculum specialist had teaching 
experience in Puerto Rico and on the mainland. She had a 
master's degree and was enrolled in a program for a second 
master's degree. 



The career and curriculum specialist at William H. Taft High 
school worked only one-quarter time. He had been a special 
education teacher for a number of years and was a work/study 
teacher, supervising students in an on-the-job training program 
at Bronx Lebanon Hospital. The small amount of time he served at 
the high school precluded his helping all the students who needed 
his services. 

The project director noted that the project lacked an office 
aide and badly needed one. She stressed that the addition of an 
office aide from the community would be particularly helpful. 
She also noted that there were not enough paraprof essionals in 
content area courses and special programs. 

DELIVERY OF SERVICES 

Project CELSIM provided various services for students. Each 
site offered E.S.L. and N.L.A. courses. Content area courses 
were taught in the native language or with E.S.L. techniques. 

The program's bilingual guidance counselor provided career 
counseling, worked with potential dropouts and counseled students 
on drug awareness, self-esteem, goal sett;ng, and conflict 
resolution, she also counseled parents on the problems of 
parent-child interaction and child abuse, and offered career 
information. The project offered career-oriented activities, 
including the use of computers and video equipment. 

Project CELSIM offered staff development activities, and 
project staff developed curricula. 
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REPORT FORMAT 

This report is organized as follows: Chapter II outlines 
the evaluation methodology; Chapter III describes the project* s 
implementation and evaluates the attainment of its implementation 
objectives; Chapter IV presents the outcomes of the project; 
a.id Chapter V offers conclusions and recommendations based on the 
findings of this evaluation. 



II. EVALUATION METHODOLOGY 



EVALUATION QUESTIONS 

The evaluation assessed two major areas: program 
implementation and outcome. Evaluation questions included the 
following: 



Process / Implementation 

How did the program select students? 

Did the project implement the instructional activities 
for developing English language proficiency as 
proposed? 

Did the project implement activities to enable students 
to increase their Spanish reading ability? 

What percentage of staff members participated in 
teacher training conferences and workshops? 

What instructional units did the curriculum resource 
specialist develop? 



Outcome 

What was the average Normal Curve Equivalent (N.C.E.) 
gain on the Language Assessment Battery (LAB)? 

What was the average gain in native language reading 
skills exhibited by program students? 

What percentage of program students passed their 
courses in mathematics, science, and social studies? 

Whet percentage of students showed a significant 
improvement in occupational aspiration? 

What were the attendance rates of participating 
students compared to those of mainstream students? 

How frequently did project students meet with the 
bilingual guidance counselor? 



EVALUATION PROCEDURES 
Sample 

An OREA field consultant interviewed the project director 
and observed classes at two of the project sites. OREA provided 
a student data form for each student. Project CELSIM returned 
111 completed forms. 

Instruments 

Instruments included an OREA-developed project director 
questionnaire and interview and observation guides for the field 
consultant's use. Project personnel completed OREA-developed 
student data forms on program participants. 

Data Collection 

A field consultant from OREA interviewed program and school 
staff and observed classes in the spring semester. OREA gave a 
questionnaire and student data forms to the project director in 
December 1988 and May 1989; completed forms and questionnaires 
were returned at the end of each semester. 

Data Analysis 

OREA used the Language Assessment Battery to assess 
improvement in English proficiency. Project CELSIM students were 
tested at grade level each spring. Students' raw scores were 
converted to Normal Carve Equivalent (N.C.E.) scores, which have 
multiple advantages over other scoring methods. They are 
standard, normalized, and form an equal interval scale. 
("Standard" indicates that the unit of measurement is a fraction 



of the standard deviation of the original distribution of raw 
scores; "normalized" refers to the fact that the scale is 
adjusted for the norm group so that its distribution has the 
shape of a normal distribution; and "equal interval scales" allow 
for legitimate aggregation or averaging of scores.) Project 
students 1 N.C.E.s indicated their standing in relation to the 
national average of 50. 

To assess the significance of students' achievement in 
English, OREA computed a correlated t-test on LAB N.C.E. scores. 
The £-test determined whether the difference between the pre- -*.nd 
posttest scores was significantly greater than would be expected 
by chance variation alone. 

To insure representative achievement data, OREA included 
only those students who had been in the program for at least five 
months and had attended classes for at least 100 school days. 
OREA extrapolated to estimate full -year scores of late-arriving 
and early-exiting students. 

Limitation s 

Since all LEP students are entitled to receive bilingual and 
E.S.L. services, OREA was unable to select an equivalent control 
group. However, the use of two sets of data, as outlined above, 
served in lieu of a control group. 
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III. EVALUATION FINDINGS: IMPLEMENTATION 



STUDENT PLACEMENT AND PROGRAMMING 

Students were eligible for the project if they scored below 
the twenty-first percentile on the Language Assessment Battery 
(LAB) .* The project targetered students diagnosed as learning 
disabled (through psychoeducational assessment by qualified 
bilingual clinicians) . 

INSTRUCTIONAL ACTIVITIES 

Project CELSIM provided instructional activities in E.S.L., 
N.L.A. content area subjects, and career education. 

English aw a Second Language 

Project CELSIM designed its E.S.L. services to provide the 
basic instructional modifications and adaptations prescribed 
within a service category, the linguistic elements necessary for 
academic success, and the activities required to develop English 
language skills. 

Theodore Roosevelt High School offered elementary, 
intermediate, and advanced level E.S.L. courses, as well as 
elementary and intermediate E.S.L. reading. James Monroe High 
School provided students with an elementary and intermediate 

* The Language Assessment Battery (LAB) was developed by the 
Board of Education of the City of New York to measure the 
English-language proficiency of non-native speakers of English in 
order to determine whether they can participate effectively in 
classes taught in English. Students scoring below the twenty- 
first percentile on the LAB are entitled to bilingual and E.S.L. 
services. 



level E.S.L. course and also one general course in E.S.L. 
reading. E.S.L. classes at William H, Taft High School were on 
three levels — beginning, intermediate, and advanced. 

The OREA field consultant observed an elementary E.S.L. 
class at James Monroe High School. The class var, held in a 
spacious and bright computer room which displayed printed 
materials in English. Materials focused on reading and writing 
skills. Computer software served as the main instructional 
material. Students worked in pairs on a reading passage. The 
teacher interacted with students, asked and answered questions, 
and gave them feedback. The teacher and students used both 
English and Spanish. 

The project director noted that more teachers specifically 
trained in E.S.L. techniques were needed. In some cases, 
bilingual teachers taught E.S.L. classes because there were no 
E.S.L. teachers available. 

Native Language Arts 

All sites offered two levels of Spanish language arts. 
Project CELSIM tried to infuse the newest learning techniques 
into these classes. By adding native language literature to the 
curricula, the project hoped to relate the course matter to 
students* lives. Project staff also used teaching techniques 
like reader response and reader logs to involve students. 

The OREA consultant observed an advanced N.L.A. class at 
James Monroe High School. Class materials were in Spanish and 
included books of short stories and legends. A tape recorder and 
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video were available. The teacher developed cu'tural activities 
with the students. 

Content Area Subjects 

Project CELSIK offered mathematics, science, and social 
studies classes. The project's mathematics curriculum inclt. ied 
fundamentals of mathematics and sequential mathematics. The 
science curriculum consisted of general science, biology, and 
health science classes. In social studies, students took global 
studies, American government, economics, and American history — 
the full sequence necessary for graduation. Teachers integrated 
lessons on the students' native cultures and histories into the 
content areas. The project director noted a need for more 
bilingual paraprof essionals to assist with content area classes. 

Career Awareness Education 

The project infused career-oriented lessons into E.S.L., 
N.L.A., and content area classes. While building language 
skills, students learned about interviews, application forms, and 
how to behave in a business office. The project developed a two- 
week lesson plan focusing on careers and introduced it into the 
E.S.L. or N.L.A. classes at each site. In the content areas, 
teacher^ made a point of relating mathematics, science , and other 
subjects to the work world. 

ftON INSTRUCTIONAL ACTIVITIES 

The project proposed noninstructional implementation 
objectives in staff and curriculum developm.it. 



Staff Development 

The objective for staff development was: 

By the end of the first year of the project, 85 percent 
of the project staff members will have participated in 
90 percent of the teacher training conferences and 
workshops. 

Project CELSIM offered staff development workshops and 
meetings designed to increase knowledge and improve skills in 
learning strategies find in teaching E.S.L. , N.L.A., and career 
education. Topics discussed during these sessions included: 
E.S.L. and N.L.A. materials, E.S.L. methodology, E.S.L. in the 
content areas, different teaching styles in bilingual classes, 
bilingual content area materials, multicultural sharing, 
communicative language, the high dropout rate fo^ Hispanic 
st-Jents, use of computer software in E . S . L. /bilingual classes, 
Lilingual computer software, and innovative video. 

Project teachers and staff found it difficult to attend 
workshops given during school hours. The project director, 
therefore, emphasized staff development on weekly site visits, 
when she would coach teachers in the classroom. 

Still, the project director reported that the coordinator 
and the resource specialists attended all of the formal staff 
development workshops. These included a three-session 
introduction to the program, four computer demonstration classes, 
and a three-session workshop on the high dropout rate for 
Hispanic students. The project met its objective for staff 
development . 
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Curriculum Development 

The program objective for career development was: 

By the end of the first year, the curriculum resource 
specialist will have developed four instructional 
units, one in each of the content areas, utilizing a 
learning strategies approach and career infusion 
technologies. 

Although the curriculum development objective called for new 
units in the content areas, the project director said that was 
not possible. The three resource specialists were only licensed 
in E.S.L. or N.L.A., and she did not want them developing 
materials outsia-?. their specialties. Therefore, Project CELSIM 
failed to meet this objective. 

At Theodore Roosevelt High School, the project staff 
developed a two-week career-oriented unit that was used in E.S.L. 
classes. The James Monroe High School resource specialist 
developed a similar unit for use in N.L.A. classes. At William 
H. Taft High School, staff developed an instructional unit on 
video careers. 
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IV. EVALUATION FINDINGS: OUTCOMES 



INSTRUCTIONAL ACTIVITIES 

The project proposed instructional objectives in E.S.L., 
N.L.A., content area subjects, and career awareness education. 

English as a Second Language 

The evaluation objective for the development of English 

language skills was: 

By June 1989, 70 percent of the participating students 
will show statistically significant gains in English 
language proficiency, as measured by the LAB. 

Project CELSIM provided complete LAB pre- and posttest 

scores for 61 students in grades nine through twelve. Because of 

the small sample and the special population targeted by this 

project, OREA could not assess the objective as proposed. 

Native Language Arts 

The evaluation objective for the development of Spanish 
reading skills was: 

As a result of participating in the program, students 
will show significant gains in Spanish reading 
achievement. 

The project submitted data for 24 students who took La 
Prueba de Lectura. The mean score was 26.3 (s.d.= 11.3). The 
posttest mean was 28.0 (s.d.= 8.9). The mean gain was 1.7 
(s.d.= 6.9). Approximately two-thirds (62.5 percent) of the 
students showed a gain. As this was more than half the sample, 
it can be concluded that Project CELSIM met its N.L.A. objective. 
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Content Area Subjects 

The evaluation objective for content area subjects was: 

At least 7 0 percent of the students enrolled in the 
content area classes will score at or above the passing 
criterion of 65. 

Overall, program students met this objective in social 

studies and business in the fall only and almost met it in 

mathematics and science in the same semester. (See Table 3.) 

Project CELSIM did not meet the content area objective. 

Career Awareness Education 

The evaluation objective for career awareness education was: 

As a result of participating in the career awareness/ 
exploration class, 70 percent of enrolled students will 
show a significant improvement in occupational 
aspiration. 

Project CELSIM administered a career exploration/ awareness 
test in November and May. Complete data were available for 35 
students. The mean posttest was 62.3 (s.d.= 17.3) , and the mean 
posttest was 67.4 (s.d.= 18.4). This gave a mean gain of 5.1 
(s.d.-8.5), which was statistically significant. The career 
awareness education objective was met. 

NON INSTRUCTIONAL ACTIVITIES 
Attendance 

The program objective for attendance was: 

As a result of participation in the program, students 1 
attendance will be significantly higher than that of 
the mainstream students. 

Since the school's attendance rate included the attendance 

of program students, OREA determined the statistical significance 
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TABLE 1 



High School 



Passing Rates in Content Area Courses 



Fall 



Content 
Area 



Number of 
Students 



Percent 
Passing 



S pring 



Number of 

Students 



Percent 
Passing 



W. Taft 



T. Roosevelt 



J . Monroe 



TOTAL 



Mathematics 


7 


71.4 


10 


90.0 


Science 


7 


57 . 1 


3 


66.7 


Social Studies 


10 


70.0 


11 


63.6 


Business 


13 


76.9 


4 


75.0 


Mathematics 


37 


67.6 


34 


41.2 


Science 


29 


69.0 


28 


75.0 


Social Studies 


40 


70.0 


42 


64.3 


Business 


21 


76. 2 


29 


65.5 


Mathematics 


30 


70.0 


26 


53.8 


Science 


24 


66.7 


13 


30.8 


Social Studies 


28 


71.4 


24 


50.0 


Business 


6 


66.7 


h 


60.0 


Mathematics 


74 


68.9 


71 


62.0 


Science 


60 


66.7 


44 


61.4 


Social Studies 


78 


70.5 


77 


59.7 


Business 


40 


75.0 


38 


65.8 



Project CELSIM students achieved the objective in social 
studies and business courses only in the fall. 




between program and school attendance through the application of 
a z-test for the significance of a proportion. This procedure 
tests whether the difference between one proportion (program 
students' attendance) and a standard proportion (the overall 
school attendance rate) is greater than can be expected by chance 
variation. 

The attendance rate of program students at William H. Taft 
High School was 75.4 percent, and the schoolwide rate was 71.5 
percent; at James Monroe High School the attendance rate for 
program students was 77.2 percent, versus 73.1 percent 
schoolwide; at Theodore Roosevelt High School the attendance rate 
for program students was 78.5 percent, versus 77.9 percent 
schoolwide . 

The z-test results (z =0.39, 0.55, and 0.10 respectively) 
indicated that the differences in attendance rates were not 
statistically significant (e<.05). Thus, the program dxd not 
mee t its attendance objective. However, the attendance rates 
were meaningfully high for this population and actually higher 
than those of mainstream students. 

Counseling 

The program objective for counseling was: 

• Participating students will meet with the bilingual 
guidance counselor at least once monthly. 

The guidance counselor met with all participating students 

each month. Counseling was a major focus of the program. 

Project CELSIM met its counseling objective. 
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V. CONCLUSIONS AND RECOMMENDATIONS 

In its first year cf operation, Project CELSIM provided 
services to 137 special education LEP students at James Monroe, 
William H. Taft, and Theodore Roosevelt High Schools in the 
Bronx. The project provided students with supplementary 

instruction in E.S.L., N.L.A., and the content areas, as well as » 
career awareness education and counseling. Project CELSIM also 
offered a variety of staff development activities and created an 
assortment of curricula. 

The proiect's philosophy was that cognitive limitation did 
not necessarily reflect lack of capacity but could have evolved 
from faulty work habits, inadequate modes of thinking, or lack of 
exposure to intensive quality instruction — all of which could be 
helped by training. 

Because of the small sample size and the specialized 
population which was targeted for this project, OREA was unable 
to assess the E.S.L. objective as proposed. Since almost two- 
thirds of the participating students showed a gain from pretest 
to posttest on La Prueba de Lectura, the project met its N.L.A. 
objective. In social studies and business in the fall, 70 
percent of participating students passed their content area 
courses. Close to 70 percent of the students passed science and 
mathematics courses in the fall. Project CELSIM did not meet the 
content area objective. The project met its career awareness 
objective. The attendance rate of program students was not 
significantly higher than that of mainstream students, therefore 

18 
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Project CELSIM did not achieve the attendance objective. The 
bilingual guidance counselor met with all the students every 
month, and Project CELSIM was successful in achieving the 
counseling objective. Counseling was a major focus and strength 
of the program. 

At least 85 percent of the project staff participated in a 
minimum of 90 percent of the training conferences and workshops, 
allowing Project CELSIM to meet the staff development objective. 

Although project staff developed curricula, they did not 
develop an instructional unit in each of the content areas as 
proposed since they were not licensed in the content areas. 
Therefore, Project CELSIM did not achieve the curriculum 
development objective. 

The conclusions, based on the findings of this evaluation 
lead to the following recommendations: 

Make objectives more realistic for the targeted 
population. 

If funds permit, increase the time devoted to the 
program by the career and curriculum specialist at 
William H. Taft High School. 



19 



— / 

ERIC 



